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1. Introduction 


To argue is to build democracy (Fuentes 2019). The capacity to reach agree- 
ments among a diverse set of social and political actors is one of the key dimen- 
sions in the governability of complex societies (Gerring and Thacker 2008). 
Argumentation is precisely the origin of legitimacy and stability of national 
democracies. However, the nature and extension of argumentation in society 
is rapidly changing. Political scientists and public participation activists have 
systematically pushed for a greater empowerment and citizen control over po- 
litical decisions. This is what is commonly known as the «deliberative turn». 
As Dryzek states: 


The essence of democracy itself is now widely taken to be deliberation, as 
opposed to voting, interest aggregation, constitutional rights, or even self- 
government. The deliberative turn represents a renewed concern with the 
authenticity of democracy: the degree to which democratic controlis substantive 
rather than symbolic, and engaged by competent citizens (Dryzek 2002, 1). 


The achievement of citizen control is not an easy task. It requires address- 
ing major practical and conceptual challenges regarding the nature, objectives 
and methods of deliberation. 
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Regarding the nature of deliberation, Chambers (2003) asserts that the es- 
sence of democratic deliberation is talk-centric. Deliberation implies a series of 
communicative and linguistic procedures that allow for a certain deliberative at- 
titude in which different actors are able to engage as peers in an exchange of rea- 
sons with the aim of reaching a shared practical judgement (Curato et al. 2017). 
According to Chilvers (2008), exercising deliberative democracy requires the 
development of deliberative competencies that allows for the orchestration of di- 
versity, difference, antagonism and uncertainty. The emergence of dissent is a 
key dimension for a pluralist view of deliberation, as dissent is view as a marker 
of different voices being heard (Marti 2017). On the other hand, quality evi- 
dence and quality reasoning are nonetheless fundamental in order to assure that 
biased or factually wrong opinions may be discarded through the deliberation 
process (Landemore 2017). 

Regarding the objectives of deliberation, Dryzek and Niemeyer (2006) ar- 
gue that probably more important than reaching consensus in the results, it is 
reaching meta-consensus. A meta-consensus means that positions are not nec- 
essarily shared, but there is consensus regarding how a decision is made and 
what is the spectrum of legitimate options and evidence. Meta-consensus may 
be normative, epistemic or of preferences (Dryzek and Niemeyer 2007). Nor- 
mative meta-consensus means that all involved values are considered legitimate. 
Epistemic meta-consensus means that all involved evidences and beliefs are 
considered trustworthy. Finally, preference meta-consensus means that there 
is agreement regarding the spectrum of legitimate outcomes and preferences of 
deliberation. For participants, reaching a meta-consensus means not only that 
they know what the other’s positions are, but also that they know why they pre- 
fer them (Niemeyer 2011). 

Regarding the methods of deliberation, and given the complexities of broad- 
ening deliberation in democracy, new ways of designing participation have aris- 
en. The redesign initiatives are often referred to as «democratic innovations». 
According to Elstub and Escobar (2019), democratic innovations are defined as 
processes or institutions thatare new to a particular governance or policy matter 
and that aim to re-imagine or deepen the role of citizens in governing through 
an extension of the opportunities of participation, deliberation and influence. 

Most of these innovations are meant for competent adult citizens that are 
deemed ready for political engagement. However, there is a need to consider 
new innovations aimed at developing the necessary deliberative competencies 
(Curato et al. 2017) and attitudes (Chilvers 2008) that are required for this to 
happen. This educational democratic innovation should gain a bigger role, espe- 
cially when considering the urgency of preparing the new generations for public 
engagement. Furthermore, there is a need to consider new innovations aimed 
at developing new analysis strategies that add representativeness (Schecter and 
Sullivan 2018) and meaningful discourse analysis and regulation (Niemeyer 
and Jennstal 2018) to existing deliberative initiatives. 

This chapter presents our experience designing and analyzing deliberation 
initiatives during 2020. In particular, we seek to present how our experience 
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with a large-scale online citizen engagement exercise called Tenemos Que Hab- 
lar de Chile [We have to talk about Chile] helped us to re-design and expand the 
Critical Debate Model (Fuentes 2011) for the context of online learning. In the 
first section, we will present a brief historical overview of the Critical Debate 
Model. In the second section, we will describe our experience with Tenemos 
Que Hablar de Chile highlighting our learnings. In the third section, we will 
describe our proposal for an online version ofthe Critical Debate Model. With 
our experience, we aim at illuminating new possibilities for the design and ana- 
lyzing of deliberation in an online environment. 


2. The Critical Debate Model 


The traditional format ofacademic or school debate, especially the competi- 
tive version of debate is a widely popularized educational practice (Freeley and 
Steinberg 2009). However, the articulation of debate as an educational prac- 
tice and debate as described by argumentation theory is still underdeveloped 
(Fuentes 2011). 

A debate is an interrogation and defense process that aims at developing 
reasoned judgment about a proposition (Freeley and Steinberg 2009). Debate 
is also perceived as a practice that accentuates antagonism and force competi- 
tion (Fuentes and Santibafiez 2011). Finally, as participants of school debates 
are often forced to accept a fixed policy position without a possibility to change 
during the deliberation process, it can also lead to a lack of genuine epistemic 
commitment that should characterize democratic deliberations. 

The Critical Debate Model (CDM) seeks to address these challenges by em- 
phasizing the dialectical nature of debate over its competitive antagonism, while 
also allowing participants to change their initial point of view utilizing arguments 
presented by their counterparts. The CDM was created by Claudio Fuentes in 
2008 based on his experience organizing the Chilean national debate tourna- 
ment from 2002 to 2005. This tournament utilized the British Parliamentary 
Debate format. Because of the previously described challenges of this format, 
stakeholders expressed dissatisfaction. That lead to a re-interpretation of the 
tournament through the application of argumentation theory and emphasizing 
the educational dimension in 2008. Thus, the Critical Debate Model was created. 

This format integrates the idea of two opposing teams (given from the tra- 
ditional debate format) and adds a third team in charge of research. It is struc- 
tured in three rounds of argumentation. In the last round, students are allowed to 
change their initial point of view and utilize evidence and arguments presented 
by the opposing team. Additionally, its evaluation and assessment methods are 
grounded in more contemporary approaches to argumentation as a dialogical 
process. The specific details of this model will be detailed later in this chapter. 

Since its creation, the Critical Debate Model has been implemented as the 
basis for the Chilean national debate tournament from 2015 to 2018. Since 2011, 
ithas also been implemented as part of the base curricula from the undergradu- 
ate program of psychology in the Universidade Federal de Pernambuco, Brazil 
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and since 2019 in the Universita di Padova, Italy as part of the Debate Tourna- 
ments for medical students. 

Despite the fact that the Critical Debate Model addresses some of the major 
concerns raised about the traditional competitive debate formats, there are still 
some challenges left to face. On the one hand, this new format emphasizes the 
dialogical component of debate, but is still centered around competition. It is 
relevant to explore non-competitive deliberation designs to compare and define 
trade-offs between the two approaches. On the other hand, this new format em- 
phasizes the educational dimension of debate, but the forms of assessment and 
feedback that is produced as part of this format is still underdeveloped. More 
audacious or comprehensive processing strategies can be explored, especially 
considering the recent advances in data sciences, particularly in the sub-field of 
natural language processing. 

Finally, this format is designed for co-located interaction and thus is not 
necessarily suited for an online environment. An online version of the Critical 
Debate Model would allow for greater flexibility and more importantly, the uti- 
lization of distributed teams across countries and continents. This has become 
a much greater issue because of the COVID-19 crisis that has impossibilities all 
co-located team activities. In this context, and drawing from our experience de- 
signing and processing a massive citizen deliberation exercise (Tenemos Que 
Hablar de Chile) we seek to present an adjusted and improved version of the 
CDM for an online age. 


3. Analyzing Tenemos Que Hablar de Chile [We have to talk about Chile] 


In 2020, our team was invited to participate in a large-scale public engage- 
ment exercise to talk about the future of Chile in the aftermath of the social 
crisis of October, 2019 that shook up the political climate of the country. The 
Tenemos Que Hablar de Chile project was developed through a partnership 
between the two most prestigious universities in Chile (Pontificia Universidad 
Católica de Chile and Universidad de Chile) with the funding of private citizens. 
The project aims at addressing the citizen’s demands for a bigger, more inclusive 
and more transparent democratic space, in which it is possible to collaboratively 
construct a vision for the future of Chile. In particular, the project seeks to de- 
velop a citizen platform for public engagement. According to the organizers, the 
objectives of this initiative are threefold: (1) To impulse a massive social conver- 
sation about the challenges of the country. (2) To promote a way of dialoguing 
that showcases our differences and allows for our reunion around them. (3) To 
systematize, represent and respond to the vision of the future by the Chilean 
society in a rigorous manner. 

Paralleling many of the concerns that have given form the «deliberative 
turn» in political science, Tenemos Que Hablar de Chile (TQHDC) defines 
its mission in terms ofthe following principles of citizen dialogues: 

e Empathy 
e Active listening 
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* Respect 

e Plurality 

e Transparency and symmetry 
e Tolerance 

e Collaboration 

e Co-responsibility 

e Value of divergence 

* Convergence as possibility. 


Of course, it is perhaps simpler to list a series of principles than to design an 
effective way to put them in practice. TQHDC canalizes all these attributes in 
three sub-initiatives. The «Citizen's consultations» initiative coordinates indi- 
vidual responses through open-ended surveys into a series of predefined social 
topics, ranging from the environment, politics, science and technology to de- 
mocracy and decentralization. The «Digital conversations» initiative organizes 
digital dialogue events (similarly to mini-publics actions) with purposely select- 
ed participants to ensure gender, regional, educational and age representative- 
ness ofthe sample. The «Chile at scale» initiative is charged with the sampling 
process that will assure representativeness for the digital dialogues. 

The digital conversations were designed by the Public Innovation Lab ofthe 
Pontificia Universidad Católica de Chile (LIPUC) and the Institute of Argu- 
mentation of the Universidad de Chile, represented by Claudio Fuentes. Each 
conversation was designed to follow a series of activities. These activities are: 

e Affective states: All conversations start by participant's expressing how they 
are currently feeling. 

* Changing Chile: Participants answer what should Chile change, maintain 
or improve 

e Priorities: Participants had to prioritize among themes that emerged dur- 
ing the previous section and decide on a topic to continue the conversation. 

e Howto bring about change: Participant's offer their point of view regarding 
how should the changes or improvements be conducted. 

e Personal commitment: Lastly, all participants had to share a personal com- 
mitment regarding their role in the transformation processes in Chile. 


4. A plan to analyze natural speech 


Asaninterdisciplinary team, we were charged with the task to create a strategy 
for the systematization and analysis ofthe data produced as part of what is likely 
the largest private citizen dialogue initiative in the history of Chile. This is not 
an easy task, especially considering that it involved massive amounts of qualita- 
tive text expressed as natural speech (thus, not formalized or coded for analysis). 

The strategy employed for the data analysis is basedin Natural Language Pro- 
cessing (NLP). NLP is a sub-field of computer and data science that addresses 
the automatic processing and analysis of natural language (unstructured and 
spontaneous expressions of language). The analysis was carried out in the pro- 
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gramming languages Python and R. Through a vast variety of techniques, such 

as tokenization, stemming, lemmatization, labeling, semantic analysis and dic- 

tionary analysis. 

We assert that the application of NLP allows fora scalable and efficient anal- 
ysis of qualitative data (compared to ‘manual’ qualitative inquiry) that main- 
tains the naturalism of dialogue events (compared to surveys). In this sense 
we observe that recent advances in data science offer a new opportunity for an 
automatic, replicable and accountable analysis and systematization strategy 
in massive online deliberative initiatives. However, it must be stated that data 
science tools can only support and enable analysis decisions, but for an analy- 
sis plan to produce meaningful and substantive results, it must be founded on 
meaningful and substantive conceptual frameworks. In our case, we based our 
analysis in theoretical integration of argumentation studies, cognitive psychol- 
ogy and future studies. 

Overall, our analysis plan was structured in three dimensions: 

e Descriptive analysis: Through readily available data science resources, we 
can describe results in terms of salient themes, key concept’s count and ex- 
ploratory visualizations in each stage of the dialogue. 

e Futurization analysis: Through the syntactic operationalization of concep- 
tual dimensions used in the description of images of the future, we can char- 
acterize collective imaginations of the Chilean future. 

e Argumentative analysis: Based on contemporary argumentation theory, 
we can analyze the type of reasoning and arguments employed during the 
conversations and infer their effects in the preferences and positions about 
what to do about Chile. 


Overall, our strategy aimed at developing a series of automated analysis in- 
dicators that were the results of an operationalization of concepts deriving from 
cognitive psychology, argumentation theory and future studies. For instance, 
through verbal markers we were able to describe the degree of agency and per- 
ceived influence in the citizen’s images of the future (Polak 1971; Taboli and 
Kapio 2018). We were able to develop indicators of semiotic distance (Zittoun 
and Gillespie 2018) through the analysis of syntactic complements and modal 
attitudes (Ballarin 2010) through verbal construction markers. The methods 
and results of this strategy plan will be published in detail once the project con- 
cludes. Beyond the potential discussions around the pertinence and reliability 
of each particular indicator, as a team we have already succeeded in the task of 
proving the feasibility of integrating different disciplinary traditions into natural 
language processing. We have observed during this process that there is much 
more that can be done when reporting deliberation results than just describing 
the expressed conclusions. 


5. Learning from Tenemos Que Hablar de Chile 
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There were three main elements of the Critical Debate Model that we sought 
to reconsider based on our experience with Tenemos Que Hablar de Chile. These 
are: competition-driven format, analysis and processing strategies and the pos- 
sibilities opened up by online deliberation. 

TQHDC is a non-competitive and expressive instance of public engage- 
ment. These sort of approaches enable citizens to express their ideas, opinions 
and feelings regarding policy issues. In contrast to other democratic innova- 
tions, such as participatory budgeting (Elstub and Escobar 2019), TQHDC 
is not consequential in nature. This means that it doesn’t produce a material 
result that is trazable and accountable. Beyond consequentiality, we observe 
that this experience lacked the form of controversiality and opposition that 
is required for people to change their minds (Leitao 2008). The lack of op- 
position or contrasting evidence that could be presented by policy makers or 
other actors can remove the epistemic conflict element that is present in the 
competitive debate models. In this sense, and contrasting to popular opinion, 
we concluded that the competitive element is a positive aspect that should be 
kept in an online CDM. 

TQHDC was designed with the analysis plan in mind. Thisis one of the major 
victories of the project. The project involved trained facilitators that registered 
all key information that would later be used for automatic processing. In prac- 
tice, this meant utilizing a SPOCA (Subject - Predicator - Object - Comple- 
ment - Adverbial) grammatical structure. Ensuring a complete syntactic unit 
proved key for exploring non-traditional analysis ideas with Natural Language 
Processing. The CDM does not involve a strong registration process and does 
not require full SPOCA annotation. This is something that should be incor- 
porated in order to open up new possibilities for analysis. The analytical ideas 
themselves could also be adapted from our TOHDC strategy and involve argu- 
mentative level and futurization level markers. 

Finally, the TOHDC project was designed as an online experience. This 
decision was purely based on the health contingency ofthe COVID pandemic. 
However, we have been able to see how an online format allows for a level of so- 
cial integration that could otherwise not be achieved. In particular, it allowed 
organizers to set quotas regarding gender, age, schooling experience and geo- 
graphical location. Each participant could take part in the event without ever 
leavingtheir house and zone of comfort. In the context of CDM, it made us real- 
ize that digitalization is more of an opportunity than a challenge. We also learnt 
that the optimal setting for an inclusive online experience is using the least pos- 
sible amount of external software. Just as face-to-face experiences can be inno- 
vatively arranged just through rules and structured actions, video call software 
is sufficient to support most of the debate experience. Requiring a single soft- 
ware makes the experience more accessible for more people and mitigates the 
effect of internet bandwidth issues. Additional software can be for information 
registration purposes, as used in TOHDC 
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In consideration of our experience with TQHDC we were able to question 
design decisions and propose an alternative to adjust and digitalize the Critical 
Debate Model. In the next section we present this proposal. 


6. Designing the Online Critical Debate Model 


This proposal is based on argumentation learning studies and research on 
argumentation skills, development of thought and knowledge construction 
(Leitao 2000; Kuhn 1991; Billig 1996). In particular, it is based on the works of 
Selma Leitão (2000, 2008). 

Leitáo's (2008) analytical procedure was designed to capture beliefs revision 
processes during argumentation. Its analysis unit consists of three main com- 
ponents (1) the argument, (2) the counter-argument and (3) the response. Both 
generaland specific cognitive operations depend on each ofthese components; 

Firstly, the argument allows for the identification of a point of view that will 
frame the argumentation process and the ideas that ground the decision over 
the point of view selection. Cognitively speaking, the argument creates a point 
of reference in relation to which an evaluation (and potential transformation) 
of reasoned perspectives may, or may not, be installed in later phases of argu- 
mentation. Epistemically, the argument captures the momentary organization 
of knowledge by an individual in a particular topic, as well as his/her doxastic 
attitudes towards that knowledge. 

Secondly, the counter-argument captures the existence of opposition voicesina 
knowledge discourse. This, in turn, introduces dialecticism that is inherent to ar- 
gumentation. Cognitively, the counter-argument represents a discourse of alterity 
that allows the individual to evaluate his/her initial position in light of contraposi- 
tion. Epistemically, the counter-argument unleashes the process of beliefs revisions. 

Finally, the response is defined as the reaction -immediate or remote- to the 
opposition substantiated in the counter-argument. Cognitively, its presence in 
argumentation marks the metacognitive awareness by the individual of the con- 
traposing positions and forces the individual to find ways of refuting them or 
to (partially or completely) accommodate his/her previous knowledge struc- 
tures to fit the new evidence. Epistemically, it’s the final deliberation instance 
in which all available evidence is laid out and the epistemic judgment is settled. 

Based on these three dialectical components of argumentation, the Critical 
Debate Model encompasses three distinct rounds or moments: 

1. Presentation: The first round is based on the presentation of arguments. It 
promotes the discursive operation of point of view exposition that is then 
used as the frame of reference for the evaluation of learning and epistemic 
transformation. 

2. Construction and counter-arguing: The second round is based on the recep- 
tion of counter-arguments and the construction of the other's point of view. 
During this stage, it is critical to utilize learning scaffolding that promotes 
the emergence of cognitive conflict and the examination of our own thought 
(i.e. metacognition). 
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3. Integration: The third round is based on the incorporation, construction, and 
argumentative collaboration. This process supports the epistemic function of 
argumentation, that is, the changing of participant’s knowledge structures. 
During this stage, initial beliefs should be contrasted with the final remarks 
in order to find indications of change. 


I must be noted that Leitào's (2008) analytical procedure was intended to 
mark the cognitive stages of knowledge transformation. The Critical Debate 
Model seeks, in turn, to transform those descriptive remarks into normative 
criteria to design successful argumentation learning experiences. 


7. Overview of the Online Critical Debate Model 
Participants 


e 2 debate teams composed by 3 participants each 
e ljudge'steam composed by 3 participants 


Note: All participants are students in case of a school debate. One member of 
the judge's team is designated as the president ofthe jury and has an active role 
as a debate presenter. 


Team roles 


* One team for the proposition 
* Oneteam CONTRA the proposition 
e One team of judges 


Note: The team of judges is in charge of evaluating the debate, presenting and 
giving a verdict. Presenting involves framing the deliberation experience, ex- 
plaining the format and participants, while also stating the relevance of the topic. 
Only in rounds 1 and 2 do teams have to comply with their role. After ending 
stage 2 and before the closing statements, teams have freedom to choose a role 
CONTRA or PRO the proposition. This means they can change their mind giv- 
en the merit of the presented arguments. 


Team member roles 


- Arguer 1(PRO): This arguer presents a brief selection of arguments that sup- 
port his/her point of view. 

- Arguer 2 (PRO): This arguer defends Arguer 1 (PRO)’s case if it has been 
weakened or refuted by the opposing team. Ifit has not, this arguer incorpo- 
rates new arguments or consolidates the previously stated ones. 
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e Arguer 1 (CONTRA): This arguer seeks to refute or weaken the case pre- 
sented by the opposing Arguer 1. This arguer doesn’t add any new arguments, 
because they are added in the second round. 

e Arguer 2 (CONTRA): This arguer defends the case presented by Arguer 
1 (CONTRA) in case they have been addressed. If they haven't been suffi- 
ciently addressed by the opposing team, this arguer has the chance to con- 
solidate the position made by his/her team member. 

e Annotators: One member of each team registers the interventions made by his/ 
her teammates and the responses made by the opposing team. At the end of 
each round, they can ask questions, make comments or ask for clarifications in 
case some aspects ofthe opposing team’s arguments were not sufficiently clear. 

e President of the jury: He/She is the presenter of the debate. He/She explains 
the relevance of the debate to the public and presents the debating parties. 
He/She afterwards helps with the flow of the debate given his/her knowl- 
edge on the mechanics and rules of the debate. 

e Judges: They fill out the evaluation form. This form is available online for 
everybody attending the debate and is then sent out to the schools (in the 
case of school debates) for further pedagogical analysis. 


Note: The roles or arguers 1 and 2 are inverted in round two. We also empha- 
size the importance of having two separate judges for issues of legitimacy and 
reliability of the results. 


Argumentation rounds 


e Firstround: Arguer 1 (PRO) > Arguer 1 (CONTRA) > 
Arguer 2 (PRO) > Arguer 2 (CONTRA) 
*. Second round: Arguer 1 (CONTRA) > Arguer 1 (PRO) > 
Arguer 2 (CONTRA) > Arguer 2 (PRO) 
e Third round: Closing statement PRO > Closing statement CONTRA. 


Time usage 


e Presentation by the president of the jury: 10 minutes. 

e First round: 5 minutes per argumentation. 

+ Timeout: 10 minutes (this includes questions, comments and clarifications 
questions). 

e Second round: 5 minutes per argumentation. 

e Timeout: 10 minutes (this includes questions, comments and clarifications 
questions). 

e Closing statements: 5 minute per team. 

e Evaluation and verdict: 10 minutes. 


Main rules 
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e Argumentative structure: All valid arguments are comprised by a point of 
view (e.g. «we believe/assert/propose that the Chilean healthcare system 
must be replaced») plus a justification (e.g. «because it is unfair by nature») 
plus evidence (e.g. «as supported by the newly published study of X founda- 
tion that states that...>). 

e Reference: All new arguments must refer to a previous argument. Defend- 
ing arguments must refer to the base proposition and attacking arguments 
must refer to the defending argument. 

e Role restriction: During the first and second round, teams are forced to de- 
fend the point of view that was assigned to them one week before the event. 

e Role liberation: Once the second round is over, teams have the liberty to 
choose any point of view, including the opposing side or one different to the 
ones exposed during the debate. Any change in point of view should be based 
on the presented evidence and arguments presented beforehand. 

e Strength of an individual argument: Judges will evaluate the strength of 
the individual argument presented by each arguer. This evaluation is made 
through an examination ofsoundness ofthe premises and conclusions made 
from the premises. 

e Strength of collective argumentation: Judges will evaluate the strength of 
the overall chain of arguments. For every pair of arguments (for and CON- 
TRA the proposition), a winner will be determined. 


Debate proposition or theme 


A debate proposition or theme is an open-ended problem that allows for both 
moral and technical examination (e.g. the Chilean healthcare system). Both 
teams are given the theme one week before the event. They are also notified about 
their role (CONTRA or for a proposition regarding the topic). For instance: 

e PRO: «Our team asserts that the Chilean healthcare system must be 
replaced». 

* CONTRA: «Ourteamasserts that the Chilean healthcare system must not 
be replaced but rather improved». 


Closing statements 


In this stage, participants are free to choose whether to maintain or change 
their initial point of view. In any case, it is expected that this decision is made in 
consideration of meaningfulreasons. A team may legitimately conclude that the 
opposing position is (fully or partially) right. However, this does not mean that 
the conceding team should not provide robust arguments for this decision. This 
is necessary for the judges to evaluate critical thinking and intellectual honesty. 

Note: Allinformation produced during the event will be registered and ana- 
lyzed. In case of school debates, this report is sent to the institutions for learn- 
ing purposes. 
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Online platform 


There are multiple channels fit to sustain an Online Critical Debate Model. 
We recommend the utilization of two separate channels for two different pur- 
poses. A video-call software (such as Zoom or Google Meet) allows for the open 
deliberation processes that occur in the first, second and third round (the latter 
being closing statements). A platform such as Zoom is ideal considering that the 
breakout rooms function allows students to meet privately while preparing dur- 
ing the timeout periods and judges to meet during the final evaluation period. 

On the other hand, judges could work live on the material in a variety of plat- 
forms. Google Documents and Spreadsheet offers a more minimalist option, while 
other platforms suchas Stormboard or Miró allows fora more designed and stylistic 
template to do all the registration, evaluation and preparation of the final documents. 
Such a platform could also be utilized by the team’s Annotators. Figure 1 shows 
an example ofa registration system that can be translated into a digital platform. 


Round 1 Argument (pro) Argument (contra) Counter (pro) Counter (contra) 
Point of The Chilean He- The Chilean He- "The regular chan- The regular chang- 
view althcare system althcare system ges made to the he- es made to the sys- 


must be replaced must be perfected althcare system are tem indicate that it 
but not replaced proofthatitmust can be perfected 


be replaced 
Justification Because itis unfair Because it has Becauseitmeans Because with each 
by nature already improved thatitis structural- change in the law 
since its inception ly broken the services get 
better 
Evidence United Nations According to Con- Arecent pollshows An article shows 


report on the most gress latest report that 70% ofthe po- that the quality of 
unequal healthca- it has been changed pulation feels dis- the health services 
resystemsinthe more than 20 times satisfied withthe has systematically 
world system improved since the 
last 10 years 


Figure 1. Example of a registration template. 


The importance of annotators 


Annotators take note of the counterpart’s arguments in the format deter- 
mined by the rules of critical debate (point of view, justification, evidence). 
Annotation involves registering statements considering a SPOCA (Subject - 
Predicator - Object - Complement — Adverbial) grammatical structure that 
will facilitate processing with NLP tools. Furthermore, the annotators, taking 
advantage of the opportunity provided by online registration, can obtain the 
arguer's approval of the annotation and immediately validate the registration. 
This avoids having to carry out extra validation procedures. 


80 


ONLINE CRITICAL DEBATE MODEL: DELIBERATION FOR THE DIGITAL AGE 


The methodology used for the annotation of the arguments is based on an 
interactive visualization (that is, writing and grouping in a digital tool) of el- 
ements of the process and elements of the debate. This visualization is done 
through a registration software (such as Stormboard). Interactive visualization 
is a powerful tool for documenting options or agreements, and keeping jointly 
prepared or individual opinions tracked visibly, without ‘losing’ them in the di- 
alogue (Ropers 2017). Additionally, applications such as Stormboard allow the 
use of templates to implement a «one idea, one card» display. This would permit 
a visual record of all opinions and keeps them visible throughout the argumen- 
tative interaction. Having visible information throughout the debate promotes 
conscious communication towards shared meaning within a group, and with 
the goal of understanding and solving problems. 


8. Argument strength evaluation 


As stated in the rules section, during a Critical Debate Model, two differ- 
ent forms of argument evaluation take place; at the individual level and at a 
group level. A group level evaluation is a comparison between the quality ofa 
pair of arguments at the individual level. Through this evaluation, we declare 
if a particular argument is stronger or weaker (« or ») than another. To con- 
duct this evaluation, a matrix with 7 columns must be prepared beforehand. 
Figure 2 represents an example Matrix in which the comparison is expressed 
by columns A, B and C. 


R Argument A Argument B Counter- C  Counter- 
argument argument 
l  Argumentl > Argument 1 < Counter 1 (pro) < Counter 1 
(PrO) (CONTRA) (CONTRA) 
2 Argument2 < Argument 2 (PRO) > Counter 2 < Counter 1 (PRO) 
(CONTRA) (CONTRA) 


Figure 2. Example of evaluation Matrix. 


Argument evaluation criteria 


All judges compare a pair of arguments based on three criteria: (1) Argumen- 
tative structure, (2) logical coherence and (3) evidential weight. 

The first criteria examined is argumentative structure. To be evaluated, all 
arguments must first demonstrate a point ofview, justification and evidence (see 
«argumentative structure» in the rules section). If one of the arguments does 
meet any of these elements, it is automatically deemed weaker. 

The second criteria examined is logical coherence. Logical coherence means 
that the point of view, justification and evidence must be meaningfully related 
amongst each other. Each justification must remain within the same thematic 
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scope as the point of view and each evidence presented must directly support 
the interpretations made in the justification. Ifan argument is less coherent than 
another, then it is deemed weaker. 

Finally, ifall previous criteria are equally met, judges will evaluate the eviden- 
tial weight of each argument. The argument that displays the more systematic, 
valid and reliable evidence is deemed stronger. Judges are encouraged to show 
how the evidential weight will be analyzing (using which markers) beforehand. 
Some examples or markers are the use of peer-reviewed articles, recent findings, 
findings made with larger populations or during large periods of time or so on. 

In the example shown in Figure 1, we can observe that all arguments meet 
the argumentative structure criteria. However, if we consider the first argument 
pair, we observe that the evidence used in Argument (CONTRA) doesn’t di- 
rectly support the justification as it doesn’t prove that the changes have made 
the healthcare system better. Because of this lack of logical coherence, the first 
argument is considered stronger. On the other hand, if we examine the last ar- 
gument pair judges may conclude that the key difference lies in the evidential 
weight (if they predetermine that perception-based evidence is weaker than aca- 
demically published external evaluations). 


Closing statements 
Each team’s closing statements are assessed in consideration of 4 

requirements: 

e Synthesis: It is expected that participants summarize the most important 
arguments and exchanges of the debate, considering both their own argu- 
ments as well as the opposing team's ones. 

e Reflection: It is expected that participants reflect on the overall debate and 
on the arguments that were essential when deciding which point ofview was 
preferred. Judges will evaluate whether the key arguments match their own 
perception of the debate. 

e  Meta-consensus: It is expected that participants can identify one form of 
meta-consensus (normative, epistemic or preference) as part of the meta- 
cognitive process of deliberation. 

e Proposal: Itis expected that teams are able to find a solution to the discussed 
conflict. If no solution is foreseen, it is expected that at least some concilia- 
tory idea is identified. 


If teams meet all four requirements, they keep all accumulated points. If 
they only meet some of them, 2596 are discounted, and ifthey don't meet any of 
them, they lose 50% of their accumulated points. 

Note: It is important to keep in mind that students are allowed to change 
their initial position during the closing statements. However, it is still expected 
that all these elements are observable. 


Final verdict 
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Debate organizers may decide to weigh each component of the debate dif- 
ferently. We recommend that the final decision is made considering each round 
independently. Each argument round can be decided in terms of the team that 
won the most arguments out of the three argument pairs (each winning argu- 
ment is a point). Finally, the closing statement is decided based on the criteria 
mentioned in the previous section. Figure 3 shows a final verdict template. 


TeamA Team B 
Round 1 2 1 
Round 2 1 2 
Round 3 -25% V 
Winner Vv 


Figure 3. Example of final verdict template. 


Ideas for the analysis of results 


As showcased from our experience with TQHDC, having a structured and 
complete registration system allows for automatic analysis to take place. In 
turn, it permits the exploration of more audacious and creative forms of analy- 
sis through the operationalization of concepts from cognitive psychology and 
argumentation theory. 

Some of the analysis ideas we are currently exploring are: 

1. Markers of agency: A determination of the degree of perceived influence/ 
agency in the policy issue discussed. 

2. Markers of semiotic distance: A determination of the degree of abstractness 
of the arguments utilized. 

3. Markers of dimensionality: A determination of the variety of dimensions 
(cultural, economic, ethical, political, etc) involved in the arguments. 

4. Classification of controversies: A classification among epistemic, normative 
and evaluative framings of the controversies. 

S. Classification of argument orientation: A classification among co-oriented 
(convergent), divergent, coordinated or subordinated arguments in consid- 
eration of how each argument is logically connected with another. 

6. Classification of points of view: A classification among modal attitudes in 
points of view, separating factic, axiological and political points of view. 

7. Forms of meta-consensus reached: A description of the forms of meta-con- 
sensus reached (or not) during the end ofthe debate, through the identifica- 
tion of normative, epistemic and preference meta-consensus markers. 


9. Discussion 


Throughout this chapter, we have presented our experience designing and 
analysing online deliberation instances. We have presented our recent work with 
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Tenemos Que Hablar de Chile (TQHDC) in order to extract insights that helped 
us re-design the Critical Debate Model for an online age. In particular, we reaf- 
firmed the need for opposition in deliberation experiences and noticed how the 
lack of consequentiality hindered the impact of TQHDC. However, we did rec- 
ognize how this project innovated regarding the role of annotation (anticipating 
automatic processing) and the exploration of new analysis options through the 
operationalization of argumentation theory and cognitive psychology. These learn- 
ings helped us to improve the Critical Debate Model and transform it to an online 
experience. Furthermore, we noticed howan online format allowed for more social 
integration and could possibilitate the use of distributed teams using participants 
from different cultural backgrounds and continental locations. This chapter then 
summarizes howan Online Critical Debate Model can be implemented in practice. 

In Chile, debate events started out with a politically naive pedagogical pre- 
tension. However, as relevant stakeholders noticed how mobilizing debate is for 
adolescents’ social consciousness and desires for a more open democracy, de- 
bate could no longer be perceived as just a traditional ‘content’ learning strategy. 
Debate and deliberation, especially when integrated into the curricula, can be 
a powerful tool to deepen political participation and to renew our compromise 
with democracy. In this context, having a more comprehensive and well-designed 
debate model was considered critical for following up on that promise. The Criti- 
cal Debate Model seeks to do just that, through a thorough conceptualization 
based on critical and dialogical developments both in psychology and philosophy. 

On the other hand, the consolidation of globalization and the unexpected 
digitalization produced by the 2020 pandemic, forced our team to think of ways 
to translate our advances into a digital format and new technologies. We have 
now started to grasp the new possibilities that emergent technologies open, 
both for design and analysis of deliberation experiences. The internet and new 
communication technologies allow students to interact with people they may 
otherwise never have met, through stylistic and pre-structured softwares that 
make the information registration process, time keeping, instant broadcasting 
and recording and many other aspects ofa debate event much more user-friendly. 
Additionally, new advances in data science, particularly regarding natural lan- 
guage processing now allow debate organisers to create automated reports and 
to explore complex new ideas for analysis that are easily scalable, transparent 
and reliable. As we have asserted, we do believe that this is possible only if the 
analysis ideas are based on meaningful theoretical models. 

Especially now, it seems vital to come up with new ways to promote online 
critical debates. Learning how to debate is learning how to argue, deliberate, 
reach agreements or at least to map our disagreements (meta-consensus). This 
ability is critical for a post-pandemic world that has already seen how western 
democracies and social cohesion are becoming weaker each year. Developing 
online debate models in developing regions is also crucial to bring about amore 
inclusive and critical globalization process. 

Despite the relevance of designing an online critical debate experience, there 
are many challenges left to address. The CDM has to find better ways to increase 
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the traceability of the arguments, that is, to register individual arguments and 
not only the overall conclusion of the debate. Individual arguments can be con- 
sidered as the raw material of deliberation and the ultimate reference point to 
determine the presence of knowledge construction. We have made some ad- 
vances in this line by including annotators, but there is still much to advance. 

In terms of pedagogical design, we still need to find ways to promote better 
argumentative processes by allowing students to metacognitively pre-assess each 
contribution and by mitigating the incidence of ill-structured, imprecise, facti- 
cally incorrect and disingenuous opinions. NLP could allow for the development 
of automated markers of poor quality arguments to let students train and be- 
come more aware of their own process. In this sense, the current design does not 
incorporate the sort of automated feedback and scaffolding needed for students 
to self-regulate and co-regulate their own argumentative practices and beliefs. 

In terms of analysis and processing, we have to still look for better ways to 
validate registered information to assure a more reliable analysis. We propose 
that applications such as Stormboard could help in this direction to create in- 
teractive visualizations of the annotation process. Having immediate validation 
helps to increase trust in the process and to mitigate the impact of the annotator 
and judges bias. The current analysis plan also needs to be consolidated and em- 
pirically tested to determine which indicators help institutions to make better 
educational decisions after the event. 

In terms of event organizing, we should find trustworthy and transparent 
mechanisms that help participants and judges to determine what are valid sourc- 
es of information and how to avoid fake or unreliable news. This will likely be- 
come a sensitive topic considering how the notion of valid news is under political 
analysis and traditional sources of information lacking the legitimacy that they 
used to have. Regarding distributed teams, we still need to test the best way to 
cope with language barriers and time zone differences in order to make a fair 
and inclusive event in everyone's minds. Cultural and geographical differences 
can also problematize the idea oftrustworthy information sources, as some par- 
ticipants and judges may attribute more epistemic or less confidence to certain 
social identities, leading up to epistemic injustice (Fricker 2008). In this sense, 
we notice that there is still a great challenge regarding how to promote epis- 
temic justice during debates and what does that mean in this particular context. 
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